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ABSTRACT 


In  the  early  1980' s  John  J.  Hopfield  developed  a 
recurrent  network  based  on  a  model  of  biological  neurons.  In 
his  model,  each  neuron  accepts  inputs  from  all  other  neurons 
in  the  network,  modifies  each  input  with  a  weight  and  converts 
their  sum  to  an  output  via  the  non-linear  sigmoid  transfer 
function.  This  output  is  then  fed  back  to  each  of  the  input 
paths  where  the  input  signals  are  updated  before  the  next 
summation.  It  has  been  proposed  that  this  network  can  be 
successfully  applied  to  the  problem  of  system  parameter 
identification  where  the  weights  are  functions  of  the  system 
states  and  the  network,  after  being  allowed  to  process  a 
continuous  block  of  system  states,  is  guaranteed  to  converge 
to  the  system  parameters.  This  thesis  explores  the  concepts  of 
network  stability  and  solution  existence  for  a  time-invariant 
system.  It  is  shown  that  the  network  will  converge  as  expected 
provided  the  steady-state  solution  falls  within  the  range  of 
values  of  the  sigmoid  transfer  function.  Experimentation  with 
the  network  when  not  all  system  states  are  measurable  revealed 
that  knowledge  of  the  actual  system  parameters  is  necessary  to 
obtain  convergence  because  of  large  error  between  the  actual 
and  estimated  system  states,  showing  that  minimization  of  this 
error  must  take  place  before  the  network  is  integrated. 
Finally,  it  is  shown  that  as  system  parameters  vary,  the 
Hopfield  network  will  track  the  parameter  changes  provided  the 
system  remains  persistently  excited  by  the  input. 
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I .    INTRODUCTION 

A.   BACKGROUND  OF  THE  NPS  AUV  II  PROJECT 

The  Naval  Postgraduate  School's  Autonomous  Underwater 
Vehicle  II  (NPS  AUV  II)  supports  the  second  generation  of 
projects  focusing  on  the  development  of  an  unmanned, 
untethered  vehicle  possessing  sufficient,  self-contained 
intelligence  to  perform  a  broad  range  of  missions  while  being 
able  to  respond  to  unplanned  situations  and  take  appropriate 
actions.  The  project  is  part  of  the  U.S.  Navy's  ongoing 
studies  of  unmanned,  sub-surface,  marine  vehicles  and  their 
usefulness  in  an  expanded  role  in  the  future  Navy.  As 
described  in  Healey  et  al . ,  [Ref.  1],  the  NPS  AUV  II  is  the 
first  of  its  kind  to  attempt  a  Mission  Planning  Expert  System 
which  will  serve  as  a  framework  within  which  all  of  the 
vehicle's  logical  operations  and  resulting  actions  will  be 
carried  out . 

The  vehicle' s  missions  might  consist  of  any  of  the  sub- 
surface tasks  currently  being  conducted  by  manned  surface  and 
sub-surface  vehicles  and  by  free-swimming  personnel.  It  would 
be  required  to  successfully  navigate  within  a  prescribed 
operating  area  while  avoiding  all  stationary  and  moving 
obstacles  but  at  the  same  time  completely  surveying  any 
objects  which  meet  the  "special  interest"  criteria  of  its 


mission  planning  system.  Its  mission  would  also  include 
storing  and  analyzing  data  before  deciding  future  courses  of 
action  appropriate  to  its  mission  and  be  capable  of  eventually 
down-loading  all  such  data  for  human  analysis.  Unplanned 
situations  might  encompass  sudden  changes  in  the  vehicle's 
operating  environment  such  as  shifts  in  current  direction  and 
speed,  the  sudden  presence  of  belligerent  animals  or  vehicles, 
faults  in  its  own  logic  or  operating  systems,  or  errors  in  its 
mission  program  (i.e.,  incorrect  navigation  information). 

The  capabilities  of  such  a  vehicle  are  far  removed  from 
those  of  any  similar  platform  in  use  today.  Such  a  vehicle  as 
NPS  AUV  II  must  ultimately  possess  complete  "knowledge"  of  its 
own  operational  abilities,  similar  to  what  humans  refer  to  as 
"motor  skills,  "  while  at  the  same  time  be  able  to  "think" 
about  various  responses  and  courses  of  action  and  decide  which 
would  best  suit  the  goals  of  its  mission.  Such  capability,  and 
the  numbingly  vast  array  of  possibilities  it  entails,  can  not 
possibly  be  programmed  into  a  computer  as  a  series  of  tasks. 
The  prospect  of  a  so-called  "thinking  machine"  is  not  so  far- 
fetched, however,  and  this  is  where  the  theory  of  Artificial 
Neural  Networks  (ANN)  may  eventually  find  application. 

B.   ARTIFICIAL  NEURAL  NETWORKS 

The  origin  of  ANN  may  be  traced  back  to  as  early  as  1943, 
when  McCulloch  and  Pitts  wrote  their  landmark  paper  "A  Logical 
Calculus  of  Ideas  Immanent  in  Nervous  Activity"   [Ref.  21. 


Since  then  scientists,  engineers,  physicists,  and  biologists 
have  been  studying  ways  to  mathematically  model  the  human 
brain's  ability  to  accept  inputs  from  many,  completely 
different  types  of  sensors,  analyze  that  data,  decide  on  a 
course  of  action,  trigger  the  proper  response  in  its 
operational  appendages,  and  learn  from  the  results  of  that 
response.  The  focus  of  the  research  efforts  in  the  field  has 
been  centered  on  a  mathematical  relationship  called  a  neuron, 
akin  to  the  biological  brain  cell  of  the  same  name,  whose 
output  is  a  weighted  summation  of  the  inputs  from  other 
neurons  and  which  is  then  used  either  as  an  input  to  other 
neurons  or  as  part  of  the  output  of  the  network  as  a  whole. 
Most  importantly,  for  application  to  "thinking  machines, "  the 
output  of  any  particular  neuron  can  be  used  in  a  feedback  loop 
to  modify  the  weights  associated  with  its  own  inputs.  This 
type  of  network,  known  as  "recurrent,"  shows  a  minimal  ability 
to  "learn"  that  a  particular  pattern  of  inputs  produces  a 
corresponding  series  of  outputs  based  on  the  values  of  the 
input  weights,  marginally  like  the  functioning  of  short-term 
memory  in  humans . 

C.   ANN  AND  NPS  AUV  II 

In  examining  the  need  for  NPS  AUV  II  to  "know"  its  own 
capabilities  and  limitations  and  the  usefulness  of  ANN  in 
"learning"  a  pattern  associated  with  a  particular  input 
environment,  a  connection  can  be  made  with  neural  networks  for 


diagnostics   [Ref.   3]   and  system  parameter  identification 

[Ref .  4]  .  It  has  been  proposed  that  a  Hopfield  network  [Ref  .5] 

[Ref.  6]  can  be  configured  in  continuous  time  to  accept  a  time 

history  of  the  state-space  response  of  a  dynamic  system 

represented  by  the  equation 

x  =Ax  +Bu  ( 1 ) 

where  x_  is  a  vector  of  state  variables,   x_  is  its  time 

derivative,  and  u_  is  the  system  input,  and  identify  the  system 
parameters  represented  by  the  matrices  A  and  B.  System 
parameter  identification  provides  the  vehicle  with  its  own 
blueprint  for  input-output  response  upon  which  it  will  base 
its  decisions  regarding  the  proper  actions  needed  to  effect 
desired  results.  By  continuously  updating  its  own  database  as 
system  parameters  change,  such  events  as  internal  faults  or 
external,  environmental  limitations  can  be  detected  and 
diagnosed. 

The  focus  of  this  thesis  is  on  exploring  the  method  of 
Shoureshi  and  Chu  [Ref.  4]  to  determine  if  it  might,  indeed, 
be  useful  for  vehicle  system  parameter  identification  in  real 
time.  The  next  chapter  provides  a  brief  history  of  ANN, 
focusing  on  the  Hopfield  recurrent  network  and  how  it  is 
adapted  by  Shoureshi  and  Chu  to  system  parameter 
identification.  Chapter  III  presents  the  results  of  stability 
investigations  of  the  Hopfield  network  formulation  and  the 
speed  of  convergence  to  expected  solution  values.  Chapter  IV 


delves  into  study  of  the  use  of  full-state  and  reduced-order 
observers  with  the  Hopfield  network  to  determine  if  the 
network  will  correctly  identify  system  parameters  when  not  all 
system  states  are  measurable.  Chapter  V  is  a  study  of  the 
ability  of  the  network  to  track  the  pattern  of  system 
parameters  as  they  vary  with  time.  Finally,  in  Chapter  VI  a 
summary  of  the  strengths  and  limitations  of  Hopfield  networks 
and  recommendations  for  future  research  in  related  fields  are 
presented. 


II.   PARAMETER  IDENTIFICATION  AND  HOPFIELD  NETWORKS 

A.  INTRODUCTION 

This  chapter  begins  with  a  brief  history  of  the  study  of 
ANN  and  presents  an  overview  of  the  first  ANN  algorithms, 
which  continue  to  provide  a  foundation  for  current  research. 
Emphasis  is  placed  on  the  development  and  theory  of  Hopfield 
networks,  not  only  because  the  Hopfield  model  is  the  basis  for 
the  research  in  this  thesis,  but  because  it  also  helped  infuse 
the  flagging  ANN  research  community  with  new  energy  in  the 
early  1980' s.  Much  more  detailed  analyses  of  the  entire 
history  and  scope  of  ANN  research,  including  that  which  has 
been  conducted  since  Hopfield' s  efforts  in  1982-1984,  can  be 
found  throughout  Wasserman  [Ref.  7]  and  NeuralWare  [Ref.  8]. 
Those  references  also  serve  to  guide  the  history  and 
background  information  in  this  chapter. 

B.  HISTORY  OF  ANN 

As  mentioned  in  Chapter  I  the  impetus  for  the  study  of 
mathematical  models  for  brain  activity  began  with  McCulloch 
and  Pitts  in  1943.  Their  subsequent  work  focused  on  the  neuron 
model  shown  in  Figure  1,  reprinted  from  Wasserman  [Ref.  7:p. 
28],  which  shows  the  neuron  modeled  as  a  summation  of  several 
weighted  inputs.  The  result  of  the  summation  is  then  compared 
to  some  threshold  value:  if  the  threshold  is  exceeded,  the 


Figure  1 .    Single  Perceptron  Network 


output  is  1;  otherwise  it  is  0.  This  basic  system,  which 
became  the  foundation  on  which  the  first  generation  of  ANN  was 
built,  is  represented  mathematically  as 

N 


1 ,   i  f  53  Wixi  >  THRESHOLD 


i  =  l 


V=  (2) 


N 


0,   if  J^  wixi<  THRESHOLD 

i  =  l 

In  the  1950' s  and  1960's  this  simple  system  was  greatly 
expanded  into  multi-layered  networks,  with  variations  being 
applied  to  such  diverse  fields  as  pattern  classification 
(Rosenblatt  [Ref.  9]),  signal  filtering  (Widrow  [Ref.  10]), 
and  macroscopic  intelligence  (Minsky  [Ref.  11])  .  All  such 
applications  were  based  on  a  learning  rule  proposed  by  Hebb 
[Ref.  12]  which  assumes  a  multi-layered  network  where  neurons 
are  interconnected  as  well  as  available  to  receive  and 
transmit  external  inputs  and  outputs.  Basically,  where  two 
neurons,  represented  by  subscripts  i  and  j,  are  connected  to 
each  other  and  to  other  neurons  in  the  network,  the  value  of 
the  weighted  connection  is  adjusted  based  on  the  values,  or 
"excitation  levels,"  of  the  two  neurons.  In  equation  form, 

W..  (t+1)  =Wij(t)  +0^  (3) 

where   Wi3     =   the  weight  of  connection  from  i  to  j 
QL  =       the  excitation  level  of  neuron  i 

Oj      =       the  excitation  level  of  neuron  j 

The  weights  are  increased,  and  the  connections  strengthened, 
each  time  the  vector  of  inputs  produces  excitation  levels  for 
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both  i  and  j  equal  to  the  binary  value  1 .  As  each  successive 
vector  of  training  inputs  is  applied,  a  pattern  of  weight 
values  emerges  where  often-used  paths  are  strengthened  well 
above  the  levels  of  little-used  paths.  Provided  the  training 
data  is  of  sufficient  quantity,  this  pattern  will 
theoretically  produce  a  desired  vector  of  output  values 
equivalent  to  those  of  the  training  sets  and  the  network  can 
be  said  to  have  learned  the  correct  response  to  a  given  input. 
In  subsequent  uses  of  Hebbian  learning,  the  manner  in 
which  the  products  of  the  two  excitation  levels  is  obtained 
has  been  altered  through  various  functions.  Wasserman  [Ref. 
7:pp.  99-100]  uses  the  term  "activation  function"  to  describe 
a  class  of  functions  where  V=F  (0)  .  The  function  F  may  be  a 
simple  threshold  function  where 

1,   if  0>  THRESHOLD 
V=  (4) 

0,   if  0<  THRESHOLD 

and  THRESHOLD  is  some  constant  value.  Later  variations  have 
modeled  F  as  a  simple  linear  function  with  or  without  some 
gain  multiplier,  while  more  recent  forms  have  sought  to 
emulate  the  actual  activity  of  a  biological  neuron.  This  last 
form  figures  prominently  in  the  work  of  Hopfield  [Ref.  5] 
[Ref.  6]  and  will  be  the  subject  of  extensive  discussion  in 
Section  E  of  this  chapter. 

The   claims   that   the  proponents   of  ANN  were  making 
regarding   the   usefulness   and  wide   application   of  their 


algorithms  led  Minsky  to  apply  rigid  mathematical  tests  to  the 
theory  of  "perceptrons, "  a  term  coined  by  Rosenblatt  [Ref.  9] 
to  describe  his  variation  of  the  McCulloch-Pitts  algorithm.  In 
their  book  Perceptrons  [Ref.  11]  Minsky  and  Papert  proved  that 
the  perceptron  algorithm  was  not  able  to  solve  the  simple, 
"exclusive-or"  (XOR)  problem,  which  is  linearly  inseparable. 
A  detailed  analysis  of  this  shortcoming  is  presented  by 
Wasserman  [Ref.  7:pp.  29-33],  who  represents  the  XOR  problem 
in  tabular  form  for  a  perceptron  consisting  of  one  neuron  and 
two  inputs,  designated  x  and  y.  Table  I  is  reproduced  here, 
where  the  values  of  x  and  y  are  binary  and  the  output,  0, 
follows  the  XOR  rule. 

TABLE  I.  THE  "EXCLUSIVE-OR"  PROBLEM 


0 

0 

0 

1 

0 

1 

0 

1 

1 

1 

1 

0 

The  equation  for  the  summation  of  the  neuron  inputs  is 

0  =  xw,  +  yw2  (5) 

where  the  output  V  of  the  neuron  takes  the  form  of  Equation 

(4)  where  THRESHOLD  can  be  any  constant  value  between  0  and  1. 

Minsky  proved  that  there  is  no  set  of  weights,  w:  and  w2,  that 

will  completely  duplicate  Table  I,  regardless  of  the  threshold 
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value.  This  revelation,  among  others,  proved  a  heavy  blow  to 
the  study  of  ANN  and  greatly  contributed  to  the  dearth  of 
research  in  the  field  throughout  the  1970' s. 

C.   HOPFIELD'S  CONTRIBUTION 

Though  limited  work  on  ANN  continued  following  Minsky'  s 
book,  the  field  was  fully  revived  in  1982  after  a  presentation 
by  John  J.  Hopfield  to  the  National  Academy  of  Sciences  in 
1982  [Ref.  5].  Hopfield  began  with  the  system  formulation 
proposed  by  McCulloch  and  Pitts,  where  the  basic  network 
consists  of  a  set  of  neurons  which  compute  the  weighted  sum  of 
the  inputs,  then  set  the  output  to  zero  or  one  depending  on 
their  relation  to  a  set  threshold  value.  What  made  the 
Hopfield  network  unique,  however,  is  the  fact  that  the  output 
of  each  neuron  is  fed  back  to  the  inputs  of  all  other  neurons. 
(In  the  original  formulation,  Hopfield  believed  a  neuron  could 
not  be  fed  back  to  itself.  This  has  since  been  abandoned  as  a 
condition.)  An  example  of  this  "recurrent"  network  is  shown  in 
Figure  2,  reprinted  from  Wasserman  [Ref.  7:p.  95].  The 
activity  of  a  neuron  is  represented  as 


1=1 


where  the  neuron  output 


1,   if  0j  >  THRESHOLD, 
3 "   0,   if  0j  <  THRESHOLD, 
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Figure  2 .    Typical  Recurrent  Network 
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Equation  (6)  is  an  expanded  version  of  the  original  algorithm, 
where  I,  is  an  external  bias  input.  This  form  of  the  equation, 
and  others  that  follow,  appear  in  a  subsequent  paper  presented 
by  Hopfield  to  the  National  Academy  of  Sciences  [Ref.  6].  The 
basic  Hopfield  learning  rule,  contrasted  with  that  of  Hebb,  is 

N    N 

AW  =VV  (2V.-1)  (2V.-1) 

£f£f  (8) 

AW,  =AW  ^ 

In  Equation  (8),  V  are  values  at  a  kth  iteration  level  in  the 
training  (learning)  process  and  AW  updates  W  from  k  to  k+1 
as  k  — >  oo.  It  can  be  seen  that  the  connection  weights  increase 
when  the  output  of  a  neuron  is  the  same  as  the  input  but  the 
weight  values  decrease  when  the  input  and  output  differ. 

D.   THE  ISSUE  OF  STABILITY 

A  major  question  which  arose  regarding  the  recurrent 
Hopfield  network  was  that  of  stability.  This  clearly  had  not 
been  a  problem  with  perceptrons  because  of  their  feed-forward, 
static  architecture.  With  the  Hopfield  net,  however,  a  proof 
was  needed  that  the  network  would  converge  to  a  stable  state 
for  all  inputs.  Building  on  work  by  Cohen  and  Grossberg  [Ref. 
13],  Hopfield  theorized  that  his  model  followed  the  activity 
of  the  bounded  energy  function  represented  by 

v   v  v  re 

1 


E  =  "4EEW  :VV "E1  V  :+E?  v  (9) 


T4- 
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where  g  is  a  set  threshold  value,  Ii  is  a  bias,  and  the  energy 
of  the  network  must  decrease  or  remain  the  same  as  it  changes 
state.  Thus,  the  change  in  energy  due  to  a  change  in  the  value 
of  the  neuron  state  V  is 


AE  =  -AV, 


.3  =  1 


(10) 


It  can  be  seen  that  the  sign  of  the  expression  in  the  brackets 
is  inversely  proportional  to  the  change  in  energy  E. 
Substituting  Equation  (6)  into  Equation  (10)  yields 


AE  =  -AV, 


E0-^ 


(id 


or,  if  Q±  is  greater  than  the  threshold  g±/  the  bracketed  value 
is  positive  and  the  output  of  neuron  i  must  change  in  the 
positive  direction  or  remain  constant.  This  means  that  AV,  can 
only  be  positive  or  zero,  so  AE  must  be  negative,  or  E  is 
decreasing.  If  0^  <  g±/  the  bracketed  value  is  negative  and  AVi 
must  be  negative  or  zero  so  again  AE  must  be  negative. 
Finally,  if  0  =  gir  AE  is  zero  and  E  remains  constant.  Since 
under  all  conditions  E  is  either  decreasing  or  unchanged,  this 
function,  which  is  bounded,  must  eventually  reach  some  bounded 
minimum  value,  proving  the  eventual  stability  of  the  network 
states . 
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E.   THE  CONTINUOUS  HOPFIELD  MODEL 

Having  proven  that  his  discrete-state  model  was  stable 
under  all  conditions,  Hopfield  sought  to  expand  the  model  to 
cover  continuous  systems,  which  represented  a  more  realistic 
application  [Ref .  6] .  The  original  model,  where  the  change  in 
states  of  a  neuron  is  represented  by  a  binary  "on-off"  firing, 
was  retained  for  the  continuous  formulation.  However,  Hopfield 
noted  that  actual  state  changes  were  non-linear  and  lagged  the 
outputs  of  other  neurons  from  which  its  inputs  were  fed.  He 
modeled  this  characteristic  as  a  simple  RC  circuit,  with  input 
capacitance  C,  neuron  resistance  R,  and  the  impedance  between 
the  output  of  neuron  j  and  input  of  neuron  i  designated  W... 
Thus, 

C :=VwiVi-_+Ii  (12) 

where 

Q^g-Uv,)  (13) 

The  input-output  relation  represented  by  the  function  g.  is 
the  non-linear  "sigmoid"  function 

g(0)=^_^  (14) 

1  +e  *" 

where  A.  is  the  learning  rate,  a  coefficient  which  determines 
the  slope  of  the  linear  portion  of  the  sigmoid  function. 
Figure  3  is  a  plot  of  the  function  for  ?i=0.1. 
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Figure  3 .    Sigmoid  Function  for  X  -   0 . 1 
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For  the  purposes  of  determining  stability  the  energy 
function  from  Equation  (9)  now  becomes 


N    K 


E  =  -iE  E  w^viv3  +  E  4-  (V1  <v)  dv  +  £  zy, 

*■   i=l  1  =  1  i=l   ^i  'S 


The  derivative  with  respect  to  time  is 


(15) 


dE  =  _^U  dV, 
dt    £i    dt 


WinV.-_+I 


(16) 


Substituting  Equations  (12)  and  (13)  into  Equation  (16), 


dE 


^--Ec^'cv.) 


'cIV,  ^ 


dt 


i=l 


^dt  y 


(17) 


Since  g1_1  (V:)  is  a  monotonically  increasing  function  and  C.  is 
positive,  then  all  terms  in  the  right-hand  side  of  Equation 
(17)  are  positive  and  dE/dt  is  negative.  If  dE/dt  =  0,  this 
implies  that  dVVdt  =  0  and  the  neuron  is  not  changing  state, 
so  a  minimum  energy  has  been  reached.  Hopf ield' s  stability 
proof,  however,  is  not  too  convincing  as  the  energy  function 
in  this  case  is  not  necessarily  bounded  and  can,  under 
circumstances  to  be  shown,  diverge. 

F.   THE  HOPFIELD  NETWORK  FOR  PARAMETER  IDENTIFICATION 

Hopf ield' s  method  of  modelling  his  network  as  the 
minimization  of  an  energy  function  was  adapted  by  Shoureshi 
and  Chu  [Ref.  4]  for  use  in  minimizing  the  equation  error  for 
system   parameter   identification.   This   has   far-reaching 
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application  to  the  control  system  field  because  having  full 
knowledge  of  a  system's  dynamics  in  the  form  of  parameter 
identification  is  essential  for  stable  control,  and  leads  to 
adaptive  systems  if  parameters  change. 

The   state-space   form   of   any   continuous   system   is 
represented  by 

x(t)  =  Ax(t)  +Bu(t)  (lfl) 

y_(t)  =  Cx(t)  V   ; 

where  x_(t)  is  a  vector  of  system  variables,  or  states;  x_(t) 

is  their  time  derivative  vector,  u_(t)  is  some  vector  of  time- 
varying  inputs  to  the  system,  and  y_(t)  is  a  vector  of 
measurable  outputs  from  the  system  response.  The  coefficients 
A,  B,  and  C,  are  matrices  which  define  the  physical 
characteristics  of  the  system.  These  matrices  may  depend  on 
time,  but  will  be  modeled  here  as  time-invariant.  The  equation 
error  associated  with  incorrect  estimates  of  the  parameters 
when  u_(t),  x_(t),  and  _x(t)  are  fully  measured,  becomes 

e(t)  =x(t)  -Ax(t)  -Bu(t)  (19) 

The  goal  of  successful  system  identification  is  to  minimize 
the  error  between  the  estimated  and  actual  system  parameters. 
A  convenient  method  is  to  minimize  a  positive-definite 
function  defined  as  the  square  of  the  equation  error  of  the 
system.  Such  a  function  might  be 
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J  =  tr  [E  (e  (t)  •  e'  (t)}]  (20) 

Because  of  the  time-variance  of  e_(t)  ,  expected  values  are  used 

to  formulate  J,  and  the  trace  of  the  matrix  of  error  values  is 
taken  as  an  average  of  the  total  error.  Thus,  after 
substitution  of  Equation  (19)  into  (20) 

J  =  E  (V  (H'H)V-  2V/H/x_  +  x/x_  }  (21) 

where 

H  =  diag  (x^  (t)  ,u/_,  (t)},   i  =  l,n;  H  e  Rr'x  (n2  +  nr)      (22) 

and 

V;  =  [a.,  b:,  ...  a:,  b±,  ...  an,  bj  (23) 

and  aif    bi   are  the  itn  rows  of  A  and  B. 

A  conceptual  and  mathematical  similarity  between  J  and  the 
energy  function  E  of  Hopf ield' s  papers  can  now  be  seen.  The 
point  is  to  minimize  the  error  function  J  to  produce  a  least- 
squares  equation  error  of  the  continuous  control  system,  much 
as  Hopfield  showed  that  his  network  minimized  the  energy 
function  E.  Furthermore,  identification  of  the  proper  system 
parameters  requires  that  minimization  of  J  be  conducted  with 
respect  to  the  vector  of  neuron  outputs  V.  Calculating  that 
partial  derivative  yields 

4^=E  {  (H'JDVl-EiH'x)  (24) 
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At  this  point,  a  direct  connection  is  made  between  the  state- 
space  form  of  the  system  dynamics  and  the  Hopfield  model. 
Equating  Equation  (25)  with  Equation  (12)  and  assuming  R.  — >  °o 
and  Ci=l, 

aj      do^    ^  (25) 

Now,  in  comparing  Equations  (25)  and  (24)  it  can  be  seen  that 

W=-E((H/H)}    I=E{(H/x)}  (26) 

The  secret  of  system  parameter  identification  using  the 
Hopfield  model  is  revealed  by  Equation  (23)  to  be  the  steady- 
state  solution  of  the  network.  Further,  the  performance  index 
gradient  in  Equation  (24)  is  contained  in  the  time  derivative 
dG/dt. 

Though  the  association  between  the  adaptation  of  the 
Hopfield  network  for  system  parameter  identification  and 
Hopfield' s  own  proof  of  the  minimization  of  a  bounded  energy 
function  has  been  shown,  it  can  not  yet  be  said  that  stability 
for  the  new  formulation  has  been  proven.  This  proof  is 
contained  in  Appendix  A  and  is  based  on  showing  the 
minimization  of  the  positive-definite  function  J.  The  proof 
shows  that  the  time  derivative  of  J  is  always  negative  and 
therefore  J  will  always  seek  a  minimum  value. 

It  is  now  clear  that  to  use  the  Hopfield  model  for  system 
parameter  identification,  one  must  collect  all  of  the  state 
variables  of  the  system  response,  as  well  as  the  input,  and 
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use  them  to  formulate  the  Hopfield  weight  matrix  W  and  bias 
matrix  I.  Then,  one  need  solve  the  system  of  first-order  ODE's 
of  Equation  (25)  for  the  vector  V.  The  solution  obtained  for 
V  will,  by  definition,  be  a  vector  containing  the  elements  of 
the  A  and  B  matrices  of  the  state-space  form  of  the  system, 
and  system  parameter  identification  will  be  achieved. 

Figure  4  is  a  block  diagram  showing  this  process.  The 
"Averaging  System"  is  a  routine  which  accepts  the  time  history 
of  the  system  input  and  its  state-space  responses  and 
formulates  matrices  W  and  I  averaged  over  a  certain  time 
interval.  This  process  was  developed  as  a  result  of  knowledge 
gained  from  experimentation  and  is  explained  in  greater  detail 
in  the  next  chapter.  The  bias  matrix  I  is  added  to  the 
negative  product  of  the  weight  matrix  W  and  the  vector  of 
previous  neuron  outputs  V.  This  sum  is  multiplied  by  a  scale 
factor  "si"  to  speed  the  response  of  the  system  and  the  result 
is  a  time  derivative  which,  when  integrated,  yields  a  vector 
of  the  weighted  summation  of  inputs  from  all  other  neurons. 
This  vector  is  operated  on  by  the  non-linear  activation 
function  g  to  produce  the  output  of  the  current  neuron. 

The  following  chapters  in  this  thesis  present  the  results 
of  extensive  experimentation  with  the  theory  of  system 
parameter  identification  using  the  Hopfield  network.  Using  a 
test  case  for  a  simple,  time-invariant,  second-order  system 
where  the  response  and  the  parameters  are  known,  the  Hopfield 
model   is   implemented  in  computer  software  to  solve  the 
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Figure  4 .    Block  Diagram  of  Continuous  Hopf ield  Network 
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characteristic  system  of  first-order  ODE's.  Experiments  are 
conducted  using  both  linear  and  non-linear  threshold  functions 
g,  and  the  issue  of  stability  of  the  network  is  thoroughly 
explored.  As  an  attempt  at  a  more  realistic  scenario,  the 
performance  of  the  network  is  evaluated  when  the  system 
response  is  not  completely  known,  that  is,  not  all  state 
variables  are  measurable.  Finally,  in  a  case  that  would  most 
certainly  arise  in  the  operating  environment  of  NPS  AUV  II, 
parameter  identification  accuracy  is  evaluated  when  the 
parameters  themselves  change  with  time. 
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III.   RESULTS  OF  THE  HOPFIELD  SIMULATION 

A.  INTRODUCTION 

In  this  thesis,  the  theory  of  Shoureshi  and  Chu  [Ref.  4] 
using  the  Hopfield  algorithm  for  parameter  identification  has 
been  applied  to  a  known,  second-order  system  herein  referred 
to  as  the  "test  case."  This  test  case  is  the  same  as  that 
investigated  in  [Ref.  4]  but  the  solutions  have  been  found 
completely  independently.  The  purpose  of  this  exercise  is  to 
investigate  the  important  aspects  of  the  Hopfield  algorithm  as 
related  to  global  stability,  speed  of  convergence,  most 
efficient  formulation,  and  accuracy  in  identifying  known 
parameters.  This  chapter  provides  a  detailed  analysis  of  this 
test  case  and  results  of  the  Hopfield  formulations  using  both 
linear  and  non-linear  transfer  functions,  the  non-linear 
function  being  exclusively  the  so-called  "sigmoid"  function. 

B.  LINEAR  TRANSFER  FUNCTION 

The  test  case  system  chosen  is  an  oscillatory  ,  second- 
order  system  with  both  states  excited  by  the  input  signal, 
u_(t)  .  The  matrices  of  constant  parameters  are 


A  = 


-.94248  12.566 
-12.566  -.9424 


8_ 

B  = 

i.o" 

_2.0_ 

(27) 
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The  equations  for  the  system  in  state-space  form  are  written 
as  follows: 


x. 


=  [A] 


' 

<^:> 

X 

L — 2  J 

[B](u) 


(28) 


where  u_=sin(t)  is  the  system  input.  This  input  function  was 
chosen  to  produce  persistent,  system  excitation.  Figure  5  is 
a  plot  of  the  state  variable  response  of  this  system  along 
with  the  input  u.  for  initial  conditions  arbitrarily  chosen  as 
xe=[l  1]  . 

Appendix  B  contains  the  computer  code  for  this  problem 
formulation,  implemented  using  MATLAB  software.  The  first  step 
in  the  main  program,  called  "neu, "  obtains  an  appropriate  time 
history  for  the  two  state  variables  and  their  time  derivatives 
from  which  the  weight  and  bias  matrices,  W  and  I,  are  formed. 
This  step  is  necessary  because  this  approach  to  system 
identification  requires  measurement  of  x,    x,    and  _u.  For  this, 

a  Runga-Kutta  second-  and  third-order  numerical  integrator, 
provided  by  the  MATLAB  software,  generates  response  data  for 
the  system  of  two,  first-order,  ordinary  differential 
equations  (ODE)  making  up  the  state-space  formulation.  The 
subroutine  "system"  contains  the  state-space  equations  where 
A  is  the  dynamics  matrix  and  B  is  the  gain  matrix.  The  MATLAB 
numerical  integrator,  a  subroutine  called  "ode23, "  operates  on 
these  equations  given  the  time  interval  specified  by  "to"  and 
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"tf",  and  by  the  initial  conditions  "xo".  For  this  formulation 
to=0.0,  tf=1.0,  and  xo= [ 1  1].  Subroutine  "ode23"  chooses  non- 
constant  time  intervals  based  on  the  speed  of  convergence  of 
the  solutions  and  returns  the  values  of  t_,  x:,    and  x^. 

A  loop  in  the  main  program  reintroduces  these  values  for 
t,  x1;  and  21?  into  the  subroutine  "system"  to  obtain  the 
corresponding  time  derivatives  of  x_i  and  x2,  designated  _f_i  anc* 
_f_2 .  The  input  u_=sin(t)  is  calculated  given  the  time  steps 
provided  by  "ode23."  Program  "neu"  formulates  the  matrices  W 
and  I  from  the  time  histories  of  xlf  x_,,  k,  k,  and  u,  as 
presented  previously.  As  the  main  program  was  first  written, 
W  and  I  were  formulated  for  each  time  step,  but  this  quickly 
proved  to  be  unworkable  as  W  is  a  singular  matrix  for  each 
time  step.  As  such,  W  does  not  exist,  so  the  steady  state 
solution 

VSS  =  -W1I/  (29) 

likewise  does  not  exist. 

To  investigate  this  problem  further,  W  was  formulated  as 
the  average  for  each  time  interval  of  state  variable 
measurement  and  the  upper  singular  values  (usv)  of  the 
eigenvalue  pairs  were  calculated  and  plotted.  Figure  6  is  a 
plot  of  the  usv'  s  of  the  eigenvalues  of  W  versus  the  number  of 
time  steps  used  to  formulate  W  before  averaging  the  elements 
to  obtain  a  final  W.  Each  curve  shows  a  pair  of  singular 
values  (the  matrix  is  symmetric)  and  it  can  be  seen  that  until 
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the  number  of  averaging  steps  exceeds  30,  one  pair  of 
eigenvalues  is  too  close  to  zero  to  ensure  the  stability  of 
the  network  solution.  From  this  point  on,  50  was  taken  as  the 
appropriate  number  of  time  steps  over  which  to  average  W.  An 
optimization  of  this  problem  was  not  attempted  but  might  be  a 
subject  of  future  research.  To  check  that  this  was  an 
acceptable  formulation  for  W,  the  steady-state  solution  for 
the  system  parameters,  Equation  (29),  was  calculated  for  each 
time  step  of  the  state-space  response.  Figure  7  shows  a  plot 
of  this  steady  state  solution  and  it  can  be  seen,  as  expected 
when  the  input  signal  is  free  of  noise,  that  within  three  time 
steps  the  proper  0SS  is  returned.  Note  that  in  the  figure,  as 
in  the  computer  code,  0  is  denoted  "th." 

Once  the  proper  number  of  averaging  steps  was  determined, 
it  was  necessary  to  reset  the  vectors  for  t_  and  _x  to  contain 
only  that  number  of  elements.  The  main  program,  as  currently 
written,  cycles  these  re-sized  vectors  through  the  subroutine 
"system"  to  obtain  their  corresponding  time  derivatives  in  the 
matrix  f .  With  this  information  the  matrices  W  and  I  are 
formulated  by  multiplying  these  column  vectors  of  _x,  _f,  and  u. 
as  specified  to  produce  each  individual  element,  which  in  turn 
is  divided  by  50.  What  results  is  a  6X6  weight  matrix  W  and  a 
6X1  bias  matrix  I  which  are  averaged  for  the  system  state  over 
roughly  the  first  one  second  of  time. 

Subroutine  "hop2"  contains  the  Hopfield  algorithm  and 
comprises  the  next  phase  of  the  computer  code.  Program  "neu" 
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calls  "hop2,"  entering  with  W  and  I  and  integrating  the  series 
of  six,  first-order,  ODE's  to  produce  a  solution  contained  in 
the  vector  0.  As  before,  the  equations  of  the  system  are  : 

d0     6   6 
_  =  -VywV+I  (30) 

The  linearity  of  this  particular  system  lies  in  V,  which  for 
this  first  test  case  is  represented  by: 

V=0  (31) 

A  scale  factor,  sl=50,  multiplies  W  and  I  to  increase  the 
response  speed. 

Program  "neu"  uses  a  simple  Euler  integration,  where  it 
was  determined  by  experimentation  that  dt=.02  produced 
acceptable  convergence  of  the  solution  to  the  final  values  of 
©provided  integration  was  carried  out  over  100  steps.  As  will 
be  discussed  in  the  section  on  non-linear  transfer  functions, 
it  was  discovered  that  the  actual,  desired  solution  for  the 
system  parameters  is  returned  in  V,  not  0.  This  was  overlooked 
at  first  because  of  the  constraint  imposed  by  Equation  (31), 
which  essentially  means  that  there  was  a  unit  transfer 
function  for  the  linear  case. 

Figure  8  shows  a  plot  of  each  individual  element  of  0 
normalized  with  respect  to  the  actual  system  parameters  versus 
the  non-dimensionalized  time  of  Euler  integration.  It  can  be 
seen  that  all  elements  converge  to  the  expected  values  within 
G)ot  =  20.  Figure  9  is  a  plot  of  the  normalized  0  vector  and 
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Figure  8.    Normalized  A  and  B  Matrix  Coefficients,  Linear 
Function 
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shows  that  convergence  to  the  actual  system  parameter  vector 
0O  occurs  within  0)ct  =  15. 

C.   NON-LINEAR,  SIGMOID,  TRANSFER  FUNCTION 

The  case  where  the  Hopfield  network  was  implemented  using 
a  linear  transfer  function  has  been  thoroughly  investigated. 
It  has  proven  useful  for  the  purposes  of  exploring  the 
important  aspects  of  stability  associated  with  this  problem. 
In  addition,  the  linearity  of  the  system  guarantees  that  a 
solution  will  be  obtained  provided  W  is  not  singular.  The  non- 
singularity  of  W  is  ensured  by  averaging  over  an  appropriate 
time  interval. 

The  network  as  described  is  not  strictly  a  Hopfield 
network,  however,  because  the  linear  transfer  function  does 
not  accurately  model  the  response  of  a  neuron.  In  Hopfield' s 
formulation  [Ref.  6]  the  transfer  function  which  converts  the 
output  of  each  neuron  is  more  accurately  represented  by  the 
non-linear  sigmoid  function  in  Equation  (14)  where  X  is  the 
learning  rate  and  is  positive  but  less  than  one.  The  plot  of 
this  sigmoid  function,  Figure  3,  shows  its  relationship  to  the 
ideal  step  change  from  zero  to  one. 

The  sigmoid  function  is  the  basis  for  the  next  set  of 
experiments  which  were  conducted  with  the  test  case  used 
previously.  Initially,  all  parameters  remained  the  same  as  for 
the  linear  network.  Additional  parameters  that  needed 
consideration  were  the  learning  rate,  X,     and  the  sigmoid 
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function  gain,  G.  Prior  to  conducting  the  experiments  it  was 
not  immediately  apparent  that  G  would  be  necessary,  but  its 
eventual  inclusion  proved  critical  and  will  be  discussed 
further . 

Figures  (10)  and  (11)  are  two  plots  of  the  sigmoid 
function  which  highlight  the  effect  of  varying  X  and  G.  It 
should  be  noted  that  in  order  to  duplicate,  and  closely 
follow,  the  test  case  used  in  [Ref.  4],  the  form  of  the 
sigmoid  function  was  altered  to 


g(0)  =G 


1  +e 


-jp, 


-  1 


(32) 


In  Figure  10,  G  is  held  constant  at  1  while  X  is  varied  from 
0.1  to  1.0.  The  function  converges  asymptotically  to  +1  and  -1 
for  all  X,  but  the  slope  of  the  linear  portion  of  the  function 
increases  dramatically  with  increasing  X.  Figure  11  shows  X 
held  constant  at  0.1  while  G  varies  from  10  to  50.  The 
asymptotes  vary  directly  with  G  with  an  equally  dramatic 
increase  in  slope  as  G  increases. 

Initially,  X  was  set  at  0.1.  As  the  necessity  of  G  was  not 
yet  recognized,  it  was  not  included,  effectively  making  G 
equal  to  1.  Figure  12  shows  the  results  of  this  run,  revealing 
that  each  of  the  coefficients  in  the  system  output  vector,  V, 
converges  within  the  range  +1  to  -1.  Experiments  with  various 
values   for  X,      as  well   as  expanding  the  time   interval   for 
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Sigmoid  Function  as  Lambda  Increases  from  0.01   to  .1 
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Output  of  Network  Using  Sigmoid  Function 
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integrating  the  network,  proved  fruitless  in  changing  this 
incorrect  "solution." 

The  reason  for  this  failure  to  converge  to  the  expected 
results  with  what  was  believed  to  be  Hopf ield' s  original 
formulation  became  apparent  once  the  network  was  analyzed 
mathematically  with  g  (0)  included.  This  system  formulation  is 
as  follows: 

d0    JL 

-a^«-XX*<°i>  +Ii  (33) 

As  t  ->  °°,  (dSi/dt)  -»  0,  so 

N 

-52wi,g(0)  +1=0  (34) 

i-i 


and 


9(0,)  =£  (W^I.  (35) 

Note,  however,  that  the  right  hand  side  of  Equation  (35) 
returns  the  steady  state  values  of  the  system  parameters  while 
the  left  hand  side  by  definition  must  vary  between  -1  and  +1. 
Thus,  the  output  g  (0)  is  constrained  to  converge  to  these 
values.  This  response  provided  the  clue  that  some  gain 
contained  within  g  (0)  ,  which  would  extend  the  range  of  the 
sigmoid  function  to  encompass  the  expected  values  of  the 
system  parameters,  is  required  for  the  existence  of  a  steady- 
state  solution . 
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When  devising  a  method  to  determine  the  proper  values  of 
G  and  X  to  use,  it  was  decided  to  maintain  the  same  unit  value 
for  the  slope  of  the  linear  section  of  the  transfer  function. 
For  unit  slope,  then,  the  derivative  with  respect  to  0  of 
Equation  (32)  must  equal  one  at  0=0,  or 


agff  =G[2(-1)  (l+e-^r't-Jte-*8)  ] 


(36) 


and 


~3B 


e=0  =  l  =G[-2(2)-2(-X)  ]  (37) 


so 


ii^\ 


and 


G  =  T  (39) 

A  value  of  G=15  was  chosen  to  encompass  the  largest  of  the 
expected  values  of  the  system  parameters,  resulting  in 
?i=0.133,  so  these  values  were  incorporated  into  the  Hopfield 
network.  Figure  13  shows  various  plots  of  the  sigmoid  function 
when  unit  slope  is  maintained  according  to  Equation  (39)  . 

Appendix  C  contains  a  listing  of  the  new  computer  code 
incorporating  the  sigmoid  transfer  function  for  network 
formulation.  The  main  program,  called  "neu2sig, "  calls  a 
reformulated  Hopfield  network  subroutine  "hop2sig, "  and 
integrates  the  series  of  equations  where  V  now  equals  g (0)  , 
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Sigmoid  Function  for  Lambda  from  0.01   to  .1,  with  G  =  2/lambd* 
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the  sigmoid  function.  As  discussed  previously,  the  system 
parameters  are  returned  in  V,  not  0,  so  a  plot  of  the  elements 
of  V,  as  the  Euler  integration  proceeds,  reveals  the  behavior 
of  the  network  as  time  advances.  Figure  14  is  a  plot  of  the 
separate  elements  of  V  normalized  with  respect  to  the 
individual,  expected  values  of  the  system  parameters,  A  and  B, 
showing  their  convergence  within  co0t  =  20.  Figure  15  is  a 
similar  plot,  except  that  V  is  plotted  as  a  vector  normalized 
with  respect  to  the  vector  of  expected  system  parameters.  This 
plot  shows  convergence  to  within  C0ot  =  15. 

D.   REMARKS 

It  has  been  shown  that  the  Hopfield  network  algorithm  can 
be  used  to  identify  the  parameters  of  a  simple,  time- 
invariant,  second-order  system  provided  that  the  expected 
values  of  the  parameters  lie  within  the  range  of  values  of  the 
non-linear  transfer  function  used  to  convert  the  output  of 
each  neuron.  The  application  of  this  method  to  real-world 
system  identification  is  limited,  however,  by  the  necessity  of 
having  all  states  of  the  system  fully  measurable.  A  more 
realistic,  and  therefore  more  important,  situation  is  one  in 
which  at  least  one  system  state  is  not  observable,  meaning 
this  state  must  be  estimated  before  the  system  parameters  may 
be  identified.  The  next  chapter  details  the  formulation  of  a 
Hopfield  network  for  the  current  second-order  test  case,  with 
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Figure  14 . 
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Figure  15.   Normalized  Output  Vector 
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the  added  complication  that  the  state  denoted  x   is  not 
measurable . 


45 


IV.   PARAMETER  IDENTIFICATION  USING  STATE  OBSERVERS 

A.  INTRODUCTION 

The  need  for  observers  in  modern  control  system  design 
where  one  or  more  states  can  not  be  measured  has  been 
thoroughly  explored  and  documented.  The  original  test  case  as 
previously  used  is  now  adapted  to  the  exploration  of  state 
observation  applied  to  the  Hopfield  network  algorithm. 
Although  the  full  states  have  been  generated  previously  and 
are  known,  this  Chapter  examines  the  case  where  only  the  state 
variable  X2  is  measurable,  while  x_i  must  be  estimated.  This 
estimated  variable,  denoted  ft  ,  will  be  used  with  an  estimated 
variable  x.2  to  formulate  W  and  I.  Finally,  it  will  be 
determined  if  the  Hopfield  network  will  correctly  identify  the 
original  system  parameters  under  these  conditions. 

B.  FULL-STATE  OBSERVER 

For  the  exploration  of  this  problem,  the  computer  code 
developed  originally  was  left  largely  intact.  However,  the 
subroutine  "system,  "  used  previously  to  generate  a  time 
history  of  the  state  variables,  has  been  replaced  with  a 
subroutine  called  "observer"  which  contains  the  code  necessary 
to  create  a  time  history  of  the  estimated  variables  x_.  and  _x2  . 
Appendix  D  contains  the  pertinent  computer  routines  for  this 
problem.  Note  that  in  the  code  x.  and  x.  are  contained  in  the 
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matrix  "xhat,"  while  the  time  derivatives  _x.  and  x_,   are 
contained  in  "xhatdot." 

Subroutine  "observer"  now  contains  a  system  of  four  first- 
order  ODE's  of  the  following  form: 


■" 

r 

,  *1 

A 

>        < 

=   [A] 

'  +  [B]{ul 


(40) 


:■: 


f=  [A  -KC 


'+  [BJ{  u)  +  [KC  ]< 


The  matrices  A0  and  B0  are  the  observer  system  parameters  which 
initially  are  set  as  exactly  equal  to  the  actual  system 
parameters  A  and  B  in  Equation  (27)  .  The  matrix  C0  is  the 
output  matrix,  which  in  this  case  is  [0  1]  .  The  purpose  of 
this  "perfect  estimation"  of  the  actual  system  parameters  is 
to  see  if  the  Hopfield  network  can  correctly  identify  the 
actual  system  parameters  based  on  state  variable  data  produced 
by  estimation.  System  identification  through  the  use  of  an 
observer  when  the  actual  system  parameters  can  not  be 
accurately  determined  will  be  explored  later  in  this  chapter. 

The  feedback  gain  matrix  K,  which  is  the  key  to  minimizing 
the  error  between  the  actual  and  estimated  states,  is  found 
using  the  dual  form  of  the  linear  quadratic  regulator 
subroutine  "lqr"  provided  by  the  MATLAB  software.  This 
subroutine  returns  the  optimal  value  of  K  such  that  the 
feedback  law 
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u  =  -Kx  (41) 

minimizes  the  cost  function 

J  =  [(x'Qx+u'RuJdt  (42) 

For  purposes  of  this  solution  Q  was  chosen  to  be  an  identity 
matrix  and  the  control  weight  R=.01,  constrained  by  the  state 
equation 

x=Ax+Bu  (43) 

The  main  program  for  this  new  problem  formulation  is 
essentially  the  same  as  for  the  fully  measurable  system  but 
was  renamed  "neuobs."  For  the  sigmoid  function  A.=  0.1  and  G=20, 
which  were  found  to  give  slightly  faster  convergence  of  the 
solution  to  the  expected  values.  The  major  difference  in  this 
main  program  is  that  W  and  I  are  calculated  using  x_.  ,  x  ,  A. 
x_2  .  As  before,  the  final  solution  is  returned  in  the  elements 
of  the  vector  V. 

Figure  16  shows  the  convergence  of  the  elements  of  V  to 
the  expected  values  of  the  actual  system  parameters.  The  speed 
of  convergence  is  essentially  unchanged  from  that  of  the 
previous  system  where  all  states  were  measurable,  as  shown  by 
the  convergence  of  all  within  the  non-dimensional  time  value 
C00t=20.  Figure  17  shows  the  normalized  vector  I  |V|  |;  again, 
the  convergence  rate  is  very  close  to  that  of  the  fully 
measurable  system. 
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Figure  16. 


Convergence  of  System  Parameters  for  Full-State 
Observer 
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Figure  17.   Normalized  Output  Vector  for  Full-State  Observer 
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The  next  step  in  exploring  the  observer  problem  was  to 
determine  if  the  parameters  of  the  system  could  be 
successfully  identified  for  that  case  where  the  actual  system 
parameters  were  not  accurately  known.  For  this  phase  of  the 
problem  the  following  observer  matrices  were  chosen  as  initial 
estimates  which  differ  substantially  from  the  actual 
parameters  of  the  test  case: 


A0  = 


-.7   10  .0 
■10.0  -.7 


2.44' 

B„  = 

o 

1.39. 

(44) 


Figure  18  is  a  plot  of  the  resulting  solution  vector,  V,  where 
each  element  has  been  normalized  over  the  expected  value  of 
its  corresponding  system  parameter,  and  it  shows  that  the 
network  does  not  converge  to  the  expected  values  of  the  actual 
system  parameters.  Table  II  shows  the  actual  values  of  the 
system  parameters,  the  values  of  the  imperfectly  estimated 
system  parameters,  and  the  steady-state  solution  of  the 
network  using  the  observer.  It  can  be  seen  that  the  steady- 
state  of  the  network  returns  something  close  to  the  value  of 
the  estimated  parameter  for  Vss(3)  but  all  other  values  differ 
substantially  from  either  the  estimated  or  actual  system 
parameters.  Since  this  data  is  inconclusive,  Figure  19  was 
generated  which  is  a  plot  of  the  actual  and  estimated  system 
state  responses.  This  plot  shows  that  the  estimation  of  x:  and 
x2  begins  to  deteriorate  rapidly  at  about  t=0.7  and  becomes  so 
severe  that  it  may  indicate  the  presence  of  instability  in  the 
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Figure  18.   Normalized  Solution  When  Parameters  Imperfectly 
Estimated,  Full  State 
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TABLE  II.  COMPARISON  BETWEEN  ACTUAL  SYSTEM  PARAMETERS 
AND  SOLUTION  USING  FULL-STATE  OBSERVER 


ss 


A/  B  Ac,  B0  Vs 

-  0.9425  -  0.7000  -  1.1368 
12.5660  10.0000  13.3263 

-12.5660  -10.0000  -10.0132 

-  0.9425  -  0.7000  -  0.5991 
1.0000  2.4400  9.8516 
2.0000  1.3900  1.6147 


observer.  The  inability  of  this  system  to  produce  an  accurate 
estimation  of  the  system  response,  which  leads  to  inaccurate 
identification  of  system  parameters,  reveals  the  need  for 
additional  study  into  a  method  for  minimizing  the  estimation 
error  of  the  system  response  before  formulation  of  the  weight 
and  bias  matrices.  Such  a  study  was  not  conducted  for  this 
thesis  but  is  deemed  essential  before  implementation  of  a 
Hopfield  network  in  a  real-world  application. 

C.   REDUCED-ORDER  OBSERVER 

The  previous  section  dealt  with  the  full-state  observer 
applied  to  the  Hopfield  network,  where  the  estimation  of  the 
entire  state  was  returned  if  even  one  state  variable  in  a 
multi-variable  system  was  not  measurable.  As  noted  in 
Friedland  [Ref.  14],  however,  it  was  desirable  to  formulate  an 
observer  which  need  only  return  estimation  of  those  state 
variables  not  actually  measured  while  omitting  the  need  to 
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estimate  variables  already  known.  A  reduced-order  observer  was 
used  for  this  purpose  of  creating  a  time  history  of  the  state 
variables  for  the  Hopfield  network  test  case. 

Friedland  presents  two  methods  of  formulating  the  reduced- 
order  observer  depending  on  the  nature  of  the  eigenvalues  of 
the  submatrix  A22.  For  the  first  method,  the  eigenvalues  of  A22 
must  be  known  to  be  negative,  or  that  the  real  parts  of  the 
poles  of  this  submatrix  lie  in  the  left  half-plane 
sufficiently  far  from  zero  to  ensure  stability  of  the  system. 
In  this  case,  the  equation  to  estimate  the  unmeasured 
variables  is  quite  straight-forward  and  takes  the  form 


(45) 

k      =A„  C"1^  +A22x_2  +  B2u. 


However,  when  the  eigenvalues  of  A22  are  not  known  or  if  A22  is 
not  stable,  Friedland  presents  a  more  general  method  to 
estimate  the  unmeasured  states.  This  second  method  was  not 
needed  for  the  test  case  examined  in  this  thesis  and  it  is  not 
anticipated  to  be  necessary  for  application  to  NPS  AUV  II,  so 
its  formulation  has  not  been  developed  here. 

Appendix  E  shows  the  computer  code  for  the  chosen  observer 
method.  The  modified  subroutine  "observer"  contains  a  system 
of  three  first-order  ODE's  consisting  of  Equations  (28)  and 
(45)  adapted  to  estimate  £  .  The  main  program  "neuobs"  selects 
x_2  and  x.  ,  retrieves  x_„  and  &  from  the  subroutine  "observer,  " 
and  uses  them  in  formulating  W  and  I.  From  there  the  network 
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algorithm  continues  as  before.  Figure  20  shows  the  individual 
elements  of  V  as  they  converge  to  the  expected  values  of  the 
system  parameters  with  a  speed  comparable  to  that  of  the  fully 
measurable  system. 

As  with  the  full-state  observer,  experiments  were 
conducted  where  the  system  parameters  were  imperfectly 
estimated  as  being  those  from  Equation  (44) .  Figure  21  shows 
that  once  again,  the  network  solution  converges  to  values 
substantially  different  from  the  actual  system  parameters  as 
shown  by  the  comparison  in  Table  III.  For  the  situation  where 
x2  is  measurable  but  x:  is  estimated,  the  steady-state  solution 
of  the  network  converges  to  the  imperfectly  estimated 
parameter  values  A0  ( 1 ,  1 )  ,  A0  ( 1 ,  2  )  ,  and  B0  ( 1 )  .  For  the  other 
three  solution  vector  elements,  convergence  to  some  values 
close  to  neither  the  actual  nor  estimated  parameter  values  has 


TABLE  III.  COMPARISON  BETWEEN  ACTUAL  PARAMETERS  AND 
SOLUTION  FOR  REDUCED-ORDER  OBSERVER 

A,  B  A0,  BD  Vss 

-  0.9425  -  0.7000  -  0.7000 
12.5660  10.0000  10.0000 

-12.5660  -10.0000  -14.5529 

-  0.9425  -  0.7000  -  1.5149 
1.0000  2.4400  2.4400 
2.0000  1.3900  12.6629 
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Figure  20.   Normalized  Solution  Vector  for  Reduced-Order 
Observer 
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Figure  21 


Normalized   Solution   Vector   for   Imperfectly 
Estimated  Parameters,  Reduced  Order 
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resulted.  Figure  22  is  a  plot  of  the  system  response  for  the 
observer  network  and  it  shows  that  the  estimation  of  x:  is  not 
very  accurate  following  t=0.2.  Again,  this  may  indicate 
instability  in  the  observer  system  and,  at  least,  requires  an 
additional  algorithm  to  minimize  the  estimation  error  before 
formulating  W  and  I. 

D.   REMARKS 

In  this  chapter  the  use  of  full-state  and  reduced-order 
observers  has  been  explored  for  use  with  the  Hopfield  network 
for  cases  where  the  states  of  a  system  can  not  be  fully 
measured  and,  therefore,  the  weight  and  bias  matrices,  W  and 
I,  can  not  be  formulated  based  on  actual  system  states.  The 
need  for  a  means  of  estimating  certain  state  responses  before 
implementing  the  control  laws  for  the  NPS  AUV  has  become 
apparent  early  in  it's  design.  When  the  surface  dynamics  of 
the  vehicle  are  considered,  it  is  found  that  two  of  the 
necessary  system  states,  yaw  rate  r  and  heading  angle  \\f,  can 
be  easily  measured  with  onboard  sensors.  However,  the  more 
subtle  system  state  called  side  slip  v,  is  not  so  easily 
measured.  For  this  situation  it  is  desired  to  adapt  an 
observer  which  will  estimate  v  with  an  acceptable  degree  of 
accuracy.  As  shown  in  this  report,  however,  an  accurate 
observer  can  not  be  developed  unless  there  is  full  knowledge 
of  the  vehicle's  system  parameters,  which  are  not  now  known 
and  will  most  certainly  be  variable  in  the  course  of  vehicle 
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Figure  22.   Actual  and  Estimated  Responses,  Reduced  Order 
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operation.  Since  the  purpose  of  this  adaptation  of  the 
Hopfield  network  is  for  identification  of  the  system 
parameters,  clearly  the  use  of  an  observer  will  be 
problematic . 
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V.  TRACKING  OF  VARYING  SYSTEM  PARAMETERS 

A.  INTRODUCTION 

An  important  situation  to  pursue  in  the  study  of  system 
parameter  identification  is  that  of  tracking  and  correctly 
identifying  system  parameters  as  they  vary  during  system 
operation.  Applied  to  the  case  of  the  NPS  AUV  II,  such  a 
situation  may  arise  when  one  of  the  vehicle's  control  surfaces 
is  lost  or  damaged.  In  this  case,  the  dynamics  of  the  vehicle 
would  certainly  change  and,  in  order  for  the  vehicle  to 
continue  to  operate,  the  mission  controller  would  have  to 
compensate  for  these  alterations  and  continue  to  function  as 
specified.  To  do  so,  however,  it  must  have  the  capability  to 
accurately  track  the  parameter  variations  in  a  timely  manner. 

In  this  chapter,  the  ability  of  the  Hopfield  network  to 
track  and  identify  such  varying  parameters  is  explored.  For 
experimentation  the  test  case  in  modified  form  is  used  where 
all  states  are  assumed  to  be  fully  measurable.  Simple  step 
changes  are  imposed  on  one  of  the  gain  parameters,  in  this 
case  [B(D],  and  the  results  are  plotted  and  examined. 

B.  RESULTS 

Appendix  F  contains  the  code  for  tracking  this  system 
parameter.  Subroutine  "system"  shows  step  changes  implemented 
as  simple  "IF-THEN"  statements,  where  [B(l)]  varies  once  each 
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second  from  t=0.0  to  5.0.  As  before,  the  system  response  in 
terms  of  its  state  variables  is  produced  using  the  MATLAB 
integration  subroutine  "ode23." 

The  main  program  "neuobsl"  divides  the  time  history  into 
one-second  intervals  corresponding  to  the  step  changes,  then 
reads  the  state  variable  values  over  those  intervals.  The 
state  variables  are  used  to  formulate  intermediate  W  and  I 
matrices,  averaged  over  the  number  of  time  steps  within  each 
interval,  and  the  Hopfield  network's  system  of  ODE's  is 
integrated  using  subroutine  "hopobs"  to  produce  a  network 
solution  corresponding  to  that  interval  of  time.  Once  all 
intermediate  network  solutions  have  been  generated,  they  are 
assembled  into  a  global  solution  covering  the  entire  five- 
second  time  period.  Figure  23  shows  a  plot  of  [V(5)],  which  is 
the  element  of  the  network  solution  which  tracks  the  parameter 
of  interest.  Overlaid  on  the  plot  is  the  actual  variation  of 
[B(l) ],  which  allows  the  speed  and  accuracy  of  the  convergence 
of  the  solution  to  be  examined.  The  figure  shows  that  for  the 
first  three  seconds  of  time,  the  solution  tracks  the  parameter 
values  after  converging  within  G)ot  =  5  0.  This  speed  of 
convergence  is  much  slower  than  for  the  case  when  the 
parameters  remain  constant,  which  for  all  experiments  was  on 
the  order  of  C0ot=20. 

Most  importantly  for  this  network  solution,  it  can  be  seen 
from  the  figure  that  during  the  fourth,  one-second  time 
interval  the  error  in  the  tracking  element  is  unacceptably 
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Tracking  of  Varying  Gain  Parameter  [B(l)] 
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large  and,  in  fact,  the  network  has  not  achieved  convergence 
to  any  value  within  O)ot  =  200.  Following  this  discrepancy, 
during  the  fifth  time  interval,  the  speed  of  convergence  and 
accuracy  of  the  solution  return  to  that  noted  for  the  first 
three  time  intervals.  The  behavior  of  the  solution  during  the 
fourth  time  interval  presents  a  serious  lapse  in  the 
performance  of  the  network  and  must  be  studied  more  closely. 

The  data  for  this  network  solution  is  presented  in  Tables 
IV  and  V  in  order  that  the  performance  of  the  network  during 
the  fourth  time  interval  may  be  more  closely  studied.  The 
eigenvalues  of  the  weight  matrix  W  shown  in  Table  IV  reveal 
that  for  the  fourth  time  interval,  two  pairs  of  poles  are 
extremely  close  to  zero  relative  to  the  other  pair  and  to  the 
pairs  of  poles  for  the  other  time  intervals.  In  addition,  the 
small  error  found  in  the  solutions  for  the  other  time 
intervals  is  also  a  product  of  lack  of  sufficient  excitation. 
For  the  parameter  tracking  problem  it  was  difficult  to  ensure 
that  W  would  be  averaged  over  the  correct  number  of  time  steps 
because  the  subroutine  "ode23"  is  a  variable-time-interval 
integrator.  This  problem  was  not  fully  explored  but  should  be 
a  subject  to  pursue  in  future  research  involving  parameter 
tracking . 

Table  V  shows  that  the  steady-state  solution  for  the 
network  element  [V(5)]  in  the  fourth  time  interval  is  equal  to 
0.1458  which  is  not  at  all  close  to  the  expected  value  for 
[B(l)]  of  0.5.  The  reason  behind  this  isolated  discrepancy  in 
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TABLE  IV.  EIGENVALUES  OF  W  WHILE  TRACKING  [B(l)] 


TIME 

EIGENVALUE 

PAIRS 

INTERVAL 

0.0  -  1.0 

.4024 

.5014 

.2679 

1.0  -  2.0 

.1347 

.0902 

1.1047 

2.0  -  3.0 

.0977 

.0475 

.7208 

3.0  -  4.0 

.0035 

.0025 

.2478 

4.0  -  5.0 

.0279 

.0291 

1.0183 

TABLE  V.  STEADY  STATE  VALUES  OF  V  (V6S  =  W"1*!') 

9044  12.5960   -12.5660   -.9425   1.1319   2.0000 

9120  12.7052   -12.5660   -.9425   5.1165   2.0000 

9310  12.5098   -12.5660   -.9425   9.9439   2.0000 

5725  13.5320   -12.5660   -.9425   0.1458   2.0000 

7734  12.1978   -12.5660   -.9425  -4.7667   2.0000 


network  performance  lies  in  the  fact  that  during  the  time 
interval  in  question,  the  gain  parameter  being  tracked  lies 
close  to  zero.  During  this  period,  the  system  is  not  being 
persistently  excited  to  the  extent  necessary  for  the 
formulation  of  an  appropriate  W,  which  instead  tends  toward 
singularity.  In  this  case,  the  network  can  not  be  expected  to 
converge  to  the  proper  solution.  An  attempt  was  made  to 
examine  an  alternate  method  of  integrating  the  system  to 
produce  the  state  response.  Instead  of  the  variable-time- 
interval  Runga-Kutta  algorithm  of  "ode23"  the  system  was 
integrated  using  a  linear,   time-invariant,   time-response 
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algorithm  with  the  system  converted  to  discrete  form.  This 
method  did  not  produce  results  substantially  different  than 
those  shown  in  Tables  IV  and  V. 

It  can  be  seen  that  in  order  for  this  network  formulation 
to  be  used  successfully  under  conditions  of  varying  system 
parameters  some  adjustment  in  the  software  must  be  designed  to 
guard  against  insufficient  excitation  by  the  system.  It  is 
apparent  that  test  signals  need  to  be  defined  to  ensure 
suitable  conditioning  of  W.  A  set  of  persistently  exciting 
input  functions  could  be  stored  which  would  be  designed  to 
maximize  the  singular  values  of  W,  which  could  be  called  by 
the  vehicle's  mission  planner  in  order  to  optimally  "test"  the 
vehicle's  behavior. 
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VI.  SUMMARY  AND  CONCLUSION 

A.   SUMMARY 

At  the  center  of  the  Naval  Postgraduate  School's  project 
to  produce  an  Autonomous  Underwater  Vehicle  (NPS  AUV  II)  is 
the  Mission  Planning  Expert  System  (MPES) ,  a  hierarchical 
system  to  control  all  operations  of  the  vehicle  while 
executing  its  planned  mission.  An  important  part  of 
implementing  such  an  expert  system  is  mapping  and  successfully 
incorporating  knowledge  of  the  vehicle  dynamics  into  a 
comprehensive  vehicle  control  algorithm.  One  way  of  achieving 
this  goal  is  through  the  application  of  Artificial  Neural 
Networks  (ANN)  to  identification  of  the  state-space  form  of 
the  vehicle's  system  parameters. 

This  thesis  has  focused  on  the  application  of  the 
continuous  form  of  the  Hopfield  network  to  system  parameter 
identification.  In  its  original  form  [Ref.  5],  Hopfield 
presented  his  network  as  a  discrete-time  model  of  a  set  of 
biological  neurons,  each  of  which  computes  the  weighted  sum  of 
the  outputs  of  all  other  neurons,  including  its  own,  then  sets 
the  output  to  zero  or  one  depending  on  whether  it  is  above  or 
below  a  set  threshold  value.  In  a  subsequent  paper  [Ref.  6], 
he  presented  a  continuous  form  of  the  model  based  on  the 
operation  of  an  electrical  RC  circuit.  In  that  model,  the 
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relation  between  the  output  of  a  neuron,  termed  its  state,  and 
the  weighted  sum  of  its  inputs  plus  a  bias  value,  was 
characterized  by  the  non-linear  sigmoid  function,  which  he 
felt  bears  a  superficial  resemblance  to  the  manner  in  which  a 
biological  neuron  changes  states. 

For  both  models  Hopfield  sought  to  prove  the  network  would 
always  converge  to  a  unique  set  of  outputs  based  on  a  distinct 
pattern  of  inputs.  To  do  this  he  likened  the  network  to  a 
bounded  energy  function  which,  as  each  neuron  changes  state, 
is  monotonically  decreasing  and  eventually  seeks  a  minimum 
value . 

Shoureshi  and  Chu  used  this  idea  of  the  minimization  of  an 
energy  function  while  studying  the  problem  of  dynamic  system 
parameter  identification.  Their  work  forms  the  basis  of  this 
thesis,  which  is  a  detailed  study  of  the  Hopfield  network  for 
parameter  identification  with  a  view  toward  implementing  it 
into  the  control  system  of  NPS  AUV  II.  A  linear,  time- 
invariant  system  was  chosen  as  a  test  case  to  see  if  the 
outputs  of  a  Hopfield  network  would  converge  to  the  known 
system  parameters,  given  that  the  weight  and  bias  matrices,  W 
and  I,  are  formulated  from  the  system  response  as  shown 
herein.  As  the  stability  of  the  network  in  this  form  was  also 
of  major  concern  and  not  directly  inferable  from  the  Hopfield 
stability  proof,  a  proof  of  stability  was  completed  and  is 
contained  in  Appendix  A.  Numerical  experiments  were  conducted 
first  with  the  system  states  fully  measurable,  then  with  the 

69 


incorporation  of  observers  to  estimate  one  state  variable,  and 
finally  assuming  the  states  were  fully  measurable  but  one 
parameter  was  varying  with  time. 

B.   CONCLUSION 

It  has  been  show  that 


1.  Hopfield  networks  can  solve  function  minimization 
quickly  in  real  time  and,  in  particular,  can  perform 
system  parameter  identification. 

2.  The  formulation  of  the  weight  matrix  W  was  not  possible 
for  each  individual  time  step  because  the  matrix  was  at 
first  singular,  meaning  that  the  steady-state  value  of 
the  system,  which  involves  the  inverse  of  W,  did  not 
exist.  By  examining  the  eigenvalues  of  W  for  each  time 
step  it  was  determined  that  acceptable  performance  was 
obtained  when  the  state  response,  used  to  formulate  W, 
was  first  averaged  over  50  time  steps. 

3.  Integration  of  the  network  following  formulation  of  the 
W  and  I  matrices  revealed  that  the  outputs  did  converge 
to  the  actual  values  of  the  system  parameters. 
Convergence  was  speeded  up  to  within  2  cycles  of  the 
natural  frequency  by  introducing  a  scale  factor  of  50 
to  multiply  W  and  I. 

4.  For  the  network  to  identify  the  actual  system 
parameters,  the  range  of  the  non-linear  sigmoid 
function  must  encompass  the  parameters'  expected 
values . 

5.  Imperfect  estimation  of  parameter  matrices  when  using 
full-state  and  reduced-order  observers  produced 
unacceptable  system-state  estimation  error  which  led  to 
incorrect  formulations  of  W  and  I  and  inaccurate 
network  solutions. 

6.  The  network  exhibited  an  acceptable  ability  to  track 
significant  changes  in  system  parameters  as  time 
progressed.  When  the  varying  gain  parameter  [B  (1)] 
dropped  to  a  value  close  to  zero,  however,  the  system 
was  not  being  persistently  excited  and  the  solution 
performance  degraded  drastically.  Inaccuracies  in 
tracking  over  all  time  intervals  pointed  out  the  need 
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for  persistent  excitation  by  the  test  signal  to  ensure 
that  W  is  well-conditioned. 


C.   RECOMMENDATIONS 

The  following  topics  are  presented  for  possible  future 
research: 


1.  The  number  of  time  steps  in  the  response  of  the  system 
over  which  to  average  W  and  I  should  be  optimized  to 
minimize  the  network  solution  time. 

2.  A  routine  should  be  included  with  the  sigmoid  function 
to  ensure  that  its  range  will  always  encompass  the 
expected  values  of  the  system  parameters  without  being 
excessively  large. 

3.  Performance  of  the  network  should  be  investigated  when 
the  system  input  signal  contains  random  noise. 

4.  The  value  of  the  gain  SI  should  be  optimized  to  ensure 
maximum  speed  of  convergence  of  the  network   solution. 

5.  Additional  experimentation  with  Hopfield  networks  and 
observers  should  be  conducted  including  routines  to 
minimize  the  error  in  system  state  estimation. 

6.  Practical  issues  in  test  signal  amplitude  and  frequency 
content  need  to  be  addressed  to  ensure  the  appropriate 
condition  of  W. 

7.  Hopfield  networks  should  be  implemented  for 
identification  of  the  actual  operating  parameters  of 
NP  S  AUV  1 1 . 
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APPENDIX  A 

PROOF  OF  STABILITY  OF  THE  HOPFIELD  NETWORK 

FOR  PARAMETER  IDENTIFICATION 


In  the  system  identification  problem,   the  estimation  of 
erroneous  parameters  leads  to  an  equation  error  of  the  kind, 

e,(t)  =H(t)Vj  -±3(t)  /  j  =  1,  n 

in  which  the  system  model  and  the  parameter  vector  so  formed 
are  obtained  as  follows. 


System  equation:    x(t)  =Ax(t)  +Bu(t)  ;  y(t)  =Cx(t) 

Parameter  vector  definition: 
V/(t)  =  [...ai  b±...]  i  =  l,n  e  <Rn(n+r) 

where  ai  and  bi  are  the  ith  row  of  the  system  parameter 
matrices,  A  and  B.  If  rhe  measurement  matrix  H(t),  is  defined 
as, 


H(t)  = 


x'  (t)u'  (t)   .  .  .0 0.  .  . 

.  .  .0.  .  .   x'  (t)u'  (t)   .  .  .0.  .  . 
.  .  .0 0.  .  .   x'  (t)u'  (t) 


;  GCRr..n(n*r> 


this  leads  to 


x(t)  =H(t)V(t)  +e(t) 


in  which  the  errors  e(t)  account  for  measurement  noise  and 
errors  due  to  parameter  mismatch. 

Definition  of  a  positive  definite  averaged  error  squared 
energy  function,  J,  leads  to 
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J=AE  (e3(t)ej(t)  } 

i-i 

and  in  terms  of  the  parameter  set  V(t), 

J=A([  H(t)V(t)  -x(t)  ]'[  H(t)V(t)  -x(t)  ]  } 

giving 

J=V  (t)A[H'(t)  ]H(t)V(t)  -2V'  (t)A  [  H'  (t)x'  (t)  ]  +  A[x(t)x(t)] 

Since  J  is  a  positive  definite  function  of  V(t),  convergence 
of  parameters  to  a  stable  set  in  which  J  is  minimum  is 
guaranteed  if  its  time  derivative  is  negative. 

The  required  stability  condition  is  then, 

4£  =  J*£(t)V(t)<o  v  v(t),  t  =  [o,~] 

dt   dV 

Such  a  condition  is  met  if, 

9j 


'-3V 

The  Hopfield  network  can  be  shown  to  meet  that  condition 
since, 

^£  =A  [H/(t)H(t)  ]V-A  [H7  (t)x(t)  ] 


or, 


4^=wv-i 


where,   W  =  A  [  H' (t )  H  (t )  ]  ,  and   I  =  A  [  H' (t )  x  (t )  ] 


Defining   __  (t)  =- 1   leads   to   the   synaptic   excitation 

dV       dt 

equation  of  Hopfield, 
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_  =  -WV  +  I  with  V  =  g(0) 

dt 

g (0)  is  always  increasing  as  (0)  increases  (i.e.  g (0)  lies  in 

the  1st  -  3rd  quadrant  in  the  0  -  V  plane),  then  V=g' (0) -S- 

dt 

with  g'  (0)  always  positive.  The  result  demonstrates  that  J 
will  seek  a  minimum  as 

V(t)  =  -g;  (0(t)  )|^(t)  , 

which  meets  the  stability  condition  shown. 

REALIZATION  OF  THE  SOLUTION 

In  spite  of  the  above,  convergence  of  d0/dt  to  0  as  t 
progresses  does  not  always  lead  to  convergence  of  V(t)  to  the 
correct  parameters,  and  one  additional  condition  is  necessary 
to  impose. 

Assume  that  0SS— H  -00,00]  and  define, 

gmax=lim  g  (0)  as  0  ->«>  and  gmin=lim  g  (0)  as  0  ->-°° 

Define  Vss  as  the  steady-state  solution  of  V(t)  as  t  become 
large . 

Define  0SS  as  the  steady-state  solution  of  0(t)  as  t  becomes 
large . 

It  follows  that  unless 

g.min  <  Vss  <  9max 


a  steady  state  solution  for  V  does  not  exist.  The  point  is 
illustrated  in  Figure  Al . 
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/////////////////////////////// 


Figure  Al .    Range   of   g,    U=0 
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APPENDIX  B 

MAIN  PROGRAM  FOR  HOPFIELD  NETWORK 

WITH  LINEAR  ACTIVATION  FUNCTION 


Main  Program  for  Hopfield  network  with  linear  activation 
function 


% 


program  neu 


to=0;  tf=l;  xo=[l  1]  ; 
[tnew,  xnew] =ode2  3 ('system' ,to,tf,xo) ; 
nl=50; 

t=tnew (1 : nl ) ; 
x=xnew ( 1 : nl , : ) ; 

for  i=l : nl 

[ f f ] =system (t (i) , x  (i,  : ) ) ;  f(i,:)=ff;  end; 
u=sin(t);  f=f/nl; 

xxl=x  (: , 1) ' *x  (: , 1) /nl; 
xx2=x (:,2) '*x(:,2) /nl; 
xxl2=x  (:, 1) '*x(:,2) /nl; 
xlu=x  (: , 1)  ' *u/nl; 
x2u=x  (  : ,2) ' *u/nl; 
uu=u' *u/nl ; 

%    Calculate  average  values  for  weight  and  bias  matrices 


W(2,2)=xx2;    W(3,3)=xxl;    W (3, 4 ) =W (1 ,  2  )  ; 
W(4,  4)  =W(2,2)  ;   W(4,6)=x2u;   W  (5,  1 )  =W  ( 1 ,  5)  ; 


W=zeros  (6, 6)  ;  I  =  zeros ( 1 , 6) ; 

W(l,l)=xxl;  W(l,2)=xxl2;  W(l,5)=xlu;  W(2,5)=x2u; 

W(2/1)=W(1,2) 

W(3,6)=W(1,5) 

W(4,3)=W(2,1) 

W(5/2)=W(2,5) 

W(5,5)=uu;  W(6,6)=uu;  W  (6,  3)  =W  (3,  6)  ;  W  (  6,  4  )  =W  (4  ,  6)  ; 

I(l)=x(:,l)'*f  (:,  1);  I  (2)=x(:,2) '  *f  (:,1);  I  (3)  =x  ( : ,  1) '  *f  (:,2); 

I  (4)=x(:,2) ' *f (:,2) ;  I (5) =u' *f ( : , 1) ;  I (6) =u' *f ( : ,2) ; 

%    Perform  Euler  integration  on  the  Hopfield  network 
%    algorithm. 

dt=.02;  int=100;  sl=50; 
tho=ones (1,6); 

for  i=2:int; 

[thdot]=hop(W, l,tho, si) ; 

th=thdot*dt+tho' ; 

tho=th' ;  thpl(i, :)=tho; 
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end; 
theta=[-. 94248  12.566  -12.566  -.94248  1  2] ; 
thvec=sqrt (theta*theta' ) ; 
for  i=l:100; 

tvec (i) =dt *i; 

thplvec (i) =sqrt (thpl (i, : ) *thpl (i, : ) ' ) ; 
end; 

wo=sqrt (.94248^2+12. 566^2); 
plot (wo*tvec, thplvec/thvec) ; 


%    Subroutine  containing  Hopfield  algorithm 

%    program  hop 

function [thdot ] =hop (W, I , th,  si ) 

for  i=l : 6 

V(i)=th(i)  ; 
end; 

V=[V(1)  V(2)  V(3)  V(4)  V(5)  V(6)]'; 
thdot=-sl*w*v+sl*i' ; 


%    Subroutine  containing  state-space  formulation  of  test  case 

%    program  system 

function [f]=system(t,x) 

u=sin (t ) ; 

A=[-. 94248  12.566;-12.566  -.94248];  B= [ 1 ; 2 ] ; 
fl=A(l, 1) *x (1) +A(1,2) *x (2) +B(1) *u; 
f2=A(2,l)  *x(l)  +A(2,2)  *x(2)+B(2)  *u; 

f=[fl  f2]; 
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APPENDIX  C 

MAIN  PROGRAM  FOR  HOPFIELD  NETWORK 

WITH  SIGMOID  ACTIVATION  FUNCTION 


%    Main  Program  for  Hopfield  network  with  sigmoid  activation 
%    function 

%    program  neu2sig 

to=0;  tf=l;  xo=[l  1] ; 

[tnew, xnew] =ode2  3 (' system' ,to,tf,xo) ; 

nl=50; 

t=tnew (1 : nl ) ; 

x=xnew (1 :nl, : ) ; 


) ;  f (i, :)=ff;  end; 


for  i=l : nl 

[ff]=system(t(i),x(i, 
u=sin(t);  f=f/nl; 

xxl=x(: , 1) '*x(:,l) /nl; 
xx2=x(: ,2) ' *x (:,2) /nl; 
xxl2=x  (:,1)  '*x(:,2)  /nl; 
xlu=x (:, 1)  ' *u/nl; 
x2u=x ( : , 2) ' *u/nl; 
uu=u' *u/nl ; 

%    Calculate  average  values  for  weight  and  bias  matrices 

W=zeros  (6, 6) ;  I  =  zeros  (1, 6) ; 

W(l,l)=xxl;  W(l,2)=xxl2;  W(l,5)=xlu;  W(2,5)=x2u; 

W(2,  1)=W(1,2) 

W(3,6)=W(1,5) 

W(4,3)=W(2,  1) 

W(5,2)=W(2,5) 

W(5,5)=uu;  W(6,6)=uu;  W  (  6,  3)  =W  (3,  6)  ;  W  (  6,  4  )  =W  (4  ,  6)  ; 

I(l)=x(:/l)'*f  (:,1);  I  (2)  =x  ( : ,  2)  '  *f  ( : ,  1)  ;  I  (3)  =x  ( : ,  1)  '  *f  ( : ,  2)  ; 

I (4)=x(:,2) '*f  (:,2) ;  I (5) =u'  *f  ( : , 1 ) ;  I (6) =u' *f ( : , 2) ; 

%    Perform  Euler  integration  on  the  Hopfield  network 
%    algorithm. 

dt=.02;  int=100;  sl=50;  lmda=0.1;  G=20; 
tho  =  ones  (1,6); 

for  i=2 : int ; 

[thdot, V] =hop2sig (W, I, tho, si, lmda,  G)  ; 
th=thdot*dt+tho' ; 


W(2,2)=xx2;    W(3,3)=xxl;    W  (3, 4 ) =W ( 1 , 2)  ; 
W(4,  4)=W(2,2)  ;   W(4,6)=x2u;   W  (5,  1 )  =W  ( 1 ,  5)  ; 


tho=th' ;  thpl  (i,  : ) =tho;  tvec (i) =dt *i; 

Vpl  (i,  :)=V  ;  thss  (i,  :  )  =  (inv(-W)  *I'  )  '  ; 
end; 
theta=[-. 94248  12.566  -12.566  -.94248  1  2]  ; 
thvec=sqrt (theta*theta'  )  ; 
for  i=l : 6; 

Wpl  (:,  i)=Vpl  (:,i)  /theta(i)  ; 
end; 

wo=sqrt (. 94  248^2+12.566^2) ; 
plot  (wo*tvec,  Wpl)  ; 


%    Subroutine  containing  Hopfield  algorithm 

%    program  hop2sig 

function [thdot , V] =hop2sig (W, I, th, si , lmda, G) 

for  i=l:6 

V(i)=G* (2/ (l+exp(-lmda*th (i)  )  ) -1)  ; 
end; 

V=[V(1)  V(2)  V(3)  V(4)  V(5)  V(6)]'; 
thdot=-sl*W*V+sl*l'  ; 

%    Subroutine  containing  state-space  formulation  of  test  case 

%    program  system 

function [f]=system(t/x) 

u=sin (t ) ; 

A=[-. 94248  12.566;-12.566  -.94248];  B= [ 1 ; 2 ]  ; 

fl=A(l,  1)  *x(l)+A(l,  2)  *x(2)+B(l)  *u; 

f2=A(2,  l)*x(l)+A(2/2)*x(2)+B(2)*u; 
f=[fl  f2]; 
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APPENDIX  D 

MAIN  PROGRAM  FOR  HOPFIELD  NETWORK 

WITH  SIGMOID  ACTIVATION  FUNCTION 

AND  FULL-STATE  OBSERVER 


%    Main  Program  for  Hopfield  network  with  sigmoid  activation 
%    function  and  full-state  observer 

%    program  neuobs 

%    Development  of  state  variable  time  history 

to=0;  tf=l;  xo=[l  1  1  1] ; 
[tnew, xnew] =ode2  3 ( ' observer' , to, tf , xo) ; 
nl=50; 

t=tnew (1 :nl ) ; 
x=xnew  ( 1 : nl ,  :  )  ; 

for  i=l :nl 

[ f f ] =observer (t (i) , x  (i,  : ) )  ;  f(i,:)=ff;  end; 
u=sin(t);  f=f  ( : , 3 : 4) /nl; 

xxl=x  (:, 3) ' *x(  : , 3) /nl; 
xx2=x  (: , 4) ' *x  (: , 4) /nl; 
xxl2=x  (  : , 3) ' *x  (: , 4) /nl; 
xlu-x (:,3) ' *u/nl; 

x2u  =  x  (  : , 4)  ' *u/nl; 
uu=u' *u/nl ; 

%    Calculate  average  values  for  weight  and  bias  matrices 


W(2,2)=xx2;    W(3,3)=xxl;    W(3,  4)  =W(1,  2)  ; 
W(4,4)=W(2,2)  ;   W(4,6)=x2u;   W  (5,  1 )  =W  ( 1 ,  5)  ; 


W=zeros  ( 6,  6) ;  I  =  zeros (1, 6) ; 

W(l,l)=xxl;  W(l/2)=xxl2;  W(l,5)=xlu;  W(2,5)=x2u; 

W(2f  1)=W(1,2) 

W(3,6)=W(1,5) 

W(4,  3)=W(2,1) 

W(5,2)=W(2,5) 

W(5,5)=uu;  W(6,6)=uu;  W  (  6,  3)  =W  (3,  6)  ;  W  (  6,  4  )  =W  (4  ,  6)  ; 

I(l)=x(:,l)'*f  (:,1);  I  (2)=x(:,2)  '  *f  <:,1);  I  (3)  =x(:,  1) '  *f  (:,2); 

I(4)=x(:,2)'*f  (:,2);  I  (5)  =u'  *f  (  : ,  1)  ;  I  (6)  =u' *f  ( :,2)  ; 

%    Perform  Euler  integration  on  the  Hopfield  network 
%    algorithm. 

dt=.02;  int=100;  sl=50;  lmda=0.1;  G=20; 
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tho=zeros (1,6); 
for  i=2 : int ; 

[thdot, V] =hopobs (W, I, tho, si, lmda,G) ; 
th=thdot*dt+tho' ; 

tho=th' ;  thpl (i, : ) =tho;  tvec (i) =dt*i; 
Vpl  (i,  :)  =V  ;  thss  (i,  :  )  =  (inv(-W)  *!')'; 
end; 
theta=[-. 94248  12.566  -12.566  -.94248  1  2]; 
thvec=sqrt (theta*theta'  ) ; 
for  i=l : 6; 

Wpl  (:  ,  i)=Vpl  (  :  ,  i)  /theta  (i)  ; 
end; 
wo  =  sqrt (,94  248"2+12.566~2);  plot  (wo* tvec, Wpl)  ; 


%    Subroutine  containing  Hopfield  algorithm 

%    program  hopobs 

function [thdot, V] =hopobs (W, I , th, si, lmda, G) 

for  i=l :  6 

V(i) =G* (2/ (l+exp(-lmda*th(i) ) )-l) ; 
end; 

V=[V(1)  V(2)  V(3)  V(4)  V(5)  V(6)]'; 
thdot=-sl*w*v+sl*i'  ; 


%    Subroutine  containing  state-space  formulation  of  test  case 
%    and  full-state  observer 

%    program  observer 

function [ f ] =ob server (t ,  x) 

u=sin (t ) ; 

A=[-. 9424  8  12.  5 66; -12. 566  -.94248];  B= [ 1 ; 2 ]  ;  C=  [  0  1]; 

Ao=[-.7  10; -10  -.7];  Bo= [2 . 4  4 ; 1 . 39]  ;  Co=[0  1]; 

%    Ao=A;  Bo=B; 

Q=eye  (2) ;  R=.01; 

xx=x(l:2)';  xhat=x (3 : 4 ) ' ; 

[K, s]=lqr (Ao,Bo,Q,R) ; 

xxdot=A*xx+B,*ru; 

xhatdot= (Ao-K' *Co) *xhat+Bo*u+K' *Co*xx; 

f=[xxdot'  xhatdot']; 


APPENDIX  E 

MAIN  PROGRAM  FOR  HOPFIELD  NETWORK 

WITH  SIGMOID  ACTIVATION  FUNCTION 

AND  REDUCED-ORDER  OBSERVER 


%    Main  Program  for  Hopfield  network  with  sigmoid  activation 
%    function  and  reduced-order  observer 

%    program  neuobs 

%    Development  of  state  variable  time  history 

to=0;  tf=l;  xo=[l  1  1]; 

[tnew, xnew] =ode2  3 ( ' observer' , to, tf , xo) ; 
nl=50; 

t=tnew (1 :nl) ; 
x=xnew (1 :nl,  :  )  ; 
for  i=l:nl 

[ f f ] =observer (t  (i)  ,  x (i,  : )  )  ; 

f (i,l)=ff (3);  f (i,2)=ff (2)  ; 
end; 
u=sin(t);  f=f/nl; 

xxl=x  (:, 3) ' *x(: , 3) /nl; 
xx2=x (: ,2) ' *x(:,2) /nl; 

xxl2=x  (  : , 3) ' *x  (  : , 2) /nl; 
xlu  =  x  (  : , 3)  ' *u/nl ; 
x2u=x  (  : , 2) ' *u/nl ; 
uu=u' *u/nl ; 

%    Calculate  average  values  for  weight  and  bias  matrices 

W=zeros  (  6,  6) ;  I  =  zeros  ( 1 , 6) ; 

W(l,l)=xxl;  W(l,2)=xxl2;  W(l,5)=xlu;  W(2,5)=x2u; 

W(2,1)=W(1,2) 

W(3/6)=W(1,5) 

W(4,3)=W(2,1) 

W(5,2)=W(2,5) 

W(5,5)=uu;  W(6,6)=uu;  W  (  6,  3)  =W  (3,  6)  ;  W  (6,  4)  =W  (4,  6)  ; 

I(l)=x(:,  1)  '*f  (:,1)  ;  I  (2)  =x  ( : ,  2)  '  *f  (:,1)  ;  I(3)=x(:,l)  '  *f  (:,2); 

I(4)=x(:f2)'*f  (:f2);    I  (5)  =u'  *f  ( : ,  1)  ;  I  (6)  =u' *f  (:,2)  ; 

%    Perform  Euler  integration  on  the  Hopfield  network 


W(2,2)=xx2;    W(3,3)=xxl;    W (3, 4 ) =W ( 1 , 2) ; 
W(4,  4)  =W(2,2)  ;   W(4,6)=x2u;   W  (5,  1 )  =W  (1 ,  5)  ; 


%    algorithm. 

dt=.02;  int=100;  sl=50;  lmda=0.1;  G=20; 
tho=zeros  (1,6)  ; 
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for  i=2:int; 

[thdot, V] =hopobs (W, I, tho, si, lmda, G) ; 

th=thdot*dt+tho' ; 

tho=th' ;  thpl (i, : ) =tho;  tvec (i) =dt *i; 

Vpl  (i,  : ) =V ;  thss  (i, : )  =  (inv (-W) *I' ) ' ; 
end; 
theta=[-. 94248  12.566  -12.566  -.94248  1  2]  ; 
thvec=sqrt (theta*theta'  )  ; 
for  i=l:6; 

Wpl  (  :  ,  i)  =Vpl  (:,i)  /theta  (i)  ;  end; 
wo=sqrt (.94  248^2+12. 566^2);  plot (wo* tvec, Wpl) ; 


%    Subroutine  containing  Hopfield  algorithm 

%    program  hopobs 

function [thdot, V] =hopobs (W, I , th, si , lmda, G) 

for  i  =  l  :  6 

V(i) =G* (2/ (l+exp(-lmda*th(i) ) ) -1) ; 
end; 

V=[V(1)  V(2)  V(3)  V(4)  V(5)  V(6)]'; 

thdot=-sl*w*v+sl*l' ; 


%    Subroutine  containing  state-space  formulation  of  test  case 
%    and  reduced-order  observer 

%    program  observer 

function [ f ] =ob server (t , x) 

u=sin (t ) ; 

A=[-. 94248  12.566;-12.566  -.94248];  B=[l;2];  C= [ 0  1]; 

Ao=[-.7  10; -10  -.7];  Bo= [2 . 4  4 ; 1 . 39] ;  Co=[0  1]; 

%    Ao=A;  Bo=B; 

Q=eye  (1) ;  R=.01; 

xx=x(l:2)/;  xlhat=x(3); 

xxdot=A*xx+B*u; 

xlhatdot=Ao (1,2) *x (2) +Ao (1,1) *xlhat+Bo(l) *u; 

f=[xxdot'  xlhatdot]; 


APPENDIX  F 

MAIN  PROGRAM  FOR  HOPFIELD  NETWORK 

WITH  SIGMOID  ACTIVATION  FUNCTION 

TRACKING  VARIATION  IN  GAIN  PARAMETER  B(l) 


% 


Main  Program  for  Hopfield  network  with  sigmoid  activation 
function  tracking  variation  in  gain  parameter  B(l) 

program  neuobsl 


to=0;  tf=l;  xo=[l  1]; 

for  a=l:5; 

if  a==l;  to=0;  tf=l 

elseif  a==2;  to=l;  tf=2 

elseif  a==3;  to=2;  tf=3 

elseif  a==4;  to=3;  tf=4 

elseif  a==5;  to=4;  tf=5 

end; 


[t , x] =ode2  3 ('system' , to, tf , xo) ; 
[nl, p] =size (t ) ;  xo=x(nl,:); 

for  i=l :nl 

[ f f ] =system (t (i) , x (i, : ) ) ;  f(i,:)=ff;  end; 
u=sin(t);  f=f/nl; 

xxl=x  (:,  1)  '  *x  (  :  ,  1)  /nl; 
xx2=x (:,2) '*x(:,2) /nl; 
xxl2=x  (  : , 1) ' *x  (:,2) /nl; 
xlu=x ( : , 1) ' *u/nl; 
x2u=x (:,2) ' *u/nl; 
uu=u' *u/nl ; 

%    Calculate  average  values  for  weight  and  bias  matrices 


W(2,2)=xx2;    W(3,3)=xxl;    W (3, 4 ) =W (1 , 2) ; 
W(4,  4)=W(2,2)  ;   W(4,6)=x2u;   W  (5,  1 )  =W  ( 1,  5)  ; 


W=zeros  (6, 6) ;  I  =  zeros  (1, 6) ; 

W(l,l)=xxl;  W(l,2)=xxl2;  W(l,5)=xlu;  W(2,5)=x2u; 

W(2,1)=W(1,2) 

W(3,6)=W(1,5) 

W(4,3)=W(2,1) 

W(5,2)=W(2,5) 

W(5,5)=uu;  W(6,6)=uu;  W  (  6,  3)  =W  (3,  6)  ;  W  (  6,  4  )  =W  (4  ,  6)  ; 

I  (l)=x  (:,l)'*f  (:,1);  I  (2)  =x  ( : ,  2)  '  *f  ( :  /l)  ;  I  (3)  =x  ( : ,  1)  '  *f  (: ,  2)  ; 

I(4)=x(:,2)'*f  (:,2);  I  (5)  =u'  *f  ( : ,  1 )  ;  I  (6)  =u'  *f  ( : ,  2)  ; 

%    Perform  Euler  integration  on  the  Hopfield  network 
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%    algorithm. 

dt=.02;  int=1000;  sl=50;  lmda=0.1;  G=20; 
tho=ones (1,6); 

for  i=2 : int ; 

[thdot , V] =hopobs (W, I , tho, si , lmda, G) ; 

th=thdot*dt+tho' ; 

tho=th' ;  thpl  (i,  :  )  =tho;  Vpl(i,:)=V;  end; 
theta=[-. 94248  12.566  -12.566  -.94248  1  2]; 
thvec  =  sqrt (theta*theta'  )  ; 
wo=sqrt (.94248^2+12. 566^2); 
tvec=dt* [1 :5000]  '  ; 

gainb  ( :  ,  a)  =Vpl  ( : ,  5)  ; 

for  ig=l:5; 

j=1000*ig; 

k=1000*ig-999; 

gb (k: j) =gainb ( : , ig) ; 
end; 


for  i=l:1000 

for  i=1001:2000 

for  i=2001:3000 

for  i=3001:4000 

for  i=4001:5000 


BB(i)=1.0;  end; 

BB(i)=5.0;  end; 

BB(i)=10.0;  end; 

BB(i)=0.5;  end; 

BB(i)=-5.0;  end; 


plot (wo*tvec, BB, wo*tvec, gb) ; 

%    Subroutine  containing  Hopfield  algorithm 

%    program  hopobs 

function [thdot, V] =hop (W, I , th, si, lmda,  G) 

for  i=l : 6 

V(i) =GA (2/ (1+exp (-lmda*th(i) ) ) -1)  ; 
end; 

V=[V(1)  V(2)  V(3)  V(4)  V(5)  V(6)]'; 

thdot=-sl*w*v+sl*l/  ; 


%    Subroutine  containing  state-space  formulation  of  test  case 
%    with  step  changes  in  gain  parameter  B(l) 

%    program  system 

function [f]=system(t,x) 


u  =  sin  (t ) ; 

A=[-. 94248  12.566; 


12.566  -.94248 


B= [ 1 ; 2 


if      t>=1.0  &  t<2.0 

elseif  t>=2.0  &  t<3.0 

elseif  t>=3.0  &  t<4 . 0 

elseif  t>=4.0  &  t<5.0 
end; 


B=[5;2]; 

B=[10;2] ; 
B=[0.5;2]; 

B=[-5.0;2] ; 


xx=x  (1:2) ' ; 
xxdot=A*xx+B*u; 
f=xxdot' ; 
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